Yaroslavl Oblast
Oreo: A Plug-in Context Reconstructor to Enhance Retrieval-Augmented Generation
Despite the remarkable capabilities of Large Language Models (LLMs) in various NLP tasks, they remain vulnerable to hallucinations due to their limited parametric knowledge and lack of domain-specific expertise. Retrieval-Augmented Generation (RAG) addresses this challenge by incorporating external document retrieval to augment the knowledge base of LLMs. In this approach, RAG retrieves document chunks from an external corpus in response to a query, which are then used as context for the downstream language model to generate an answer. However, these retrieved knowledge sources often include irrelevant or erroneous information, undermining the effectiveness of RAG in downstream tasks. To overcome this limitation, we introduce a compact, efficient, and pluggable module designed to refine external knowledge sources before feeding them to the generator. The module reconstructs retrieved content by extracting the most relevant and supportive information and reorganising it into a concise, query-specific format. Through a three-stage training paradigm - comprising supervised fine-tuning, contrastive multi-task learning, and reinforcement learning-based alignment - it prioritises critical knowledge and aligns it with the generator's preferences. This method enables LLMs to produce outputs that are more accurate, reliable, and contextually appropriate.
Reverse Modeling in Large Language Models
Yu, Sicheng, Xu, Yuanchen, Du, Cunxiao, Zhou, Yanying, Qiu, Minghui, Sun, Qianru, Zhang, Hao, Wu, Jiawei
Humans are accustomed to reading and writing in a forward manner, and this natural bias extends to text understanding in auto-regressive large language models (LLMs). This paper investigates whether LLMs, like humans, struggle with reverse modeling, specifically with reversed text inputs. We found that publicly available pre-trained LLMs cannot understand such inputs. However, LLMs trained from scratch with both forward and reverse texts can understand them equally well during inference. Our case study shows that different-content texts result in different losses if input (to LLMs) in different directions -- some get lower losses for forward while some for reverse. This leads us to a simple and nice solution for data selection based on the loss differences between forward and reverse directions. Using our selected data in continued pretraining can boost LLMs' performance by a large margin across different language understanding benchmarks.
- North America > United States (0.14)
- Europe > Russia > Central Federal District > Yaroslavl Oblast > Yaroslavl (0.06)
- Europe > Hungary (0.05)
- (7 more...)
- Media > Film (1.00)
- Information Technology (0.68)
- Leisure & Entertainment > Games > Computer Games (0.34)
Russia, Ukraine trade drone attacks in renewed escalation
Russia has launched several strikes across Ukraine, killing at least five people and wounding several, in an attack that appeared to target energy infrastructure. Ukraine also launched a drone attack on Russia's central region of Saratov, injuring four. The exchange began around midnight on Sunday and continued beyond daybreak on Monday. Ukraine's air force reported multiple groups of Russian drones moving towards its eastern, northern, southern, and central regions, followed by numerous cruise and ballistic missiles. Authorities in at least six Ukrainian regions said blasts had been heard.
- Asia > Russia (1.00)
- Europe > Russia > Volga Federal District > Saratov Oblast > Saratov (0.33)
- Europe > Ukraine > Kyiv Oblast > Kyiv (0.11)
- (4 more...)
Authorship attribution for Differences between Literary Texts by Bilingual Russian-French and Non-Bilingual French Authors
Do bilingual Russian-French authors of the end of the twentieth century such as Andre\"i Makine, Val\'ery Afanassiev, Vladimir F\'edorovski, Iegor Gran, Luba Jurgenson have common stylistic traits in the novels they wrote in French? Can we distinguish between them and non-bilingual French writers' texts? Is the phenomenon of interference observable in French texts of Russian authors? This paper applies authorship attribution methods including Support Vector Machine (SVM), $K$-Nearest Neighbors (KNN), Ridge classification, and Neural Network to answer these questions.
- Asia > Russia (0.14)
- Europe > Switzerland > Vaud > Lausanne (0.04)
- North America > United States > Illinois > Cook County > Evanston (0.04)
- (5 more...)
A Light in the Dark: Deep Learning Practices for Industrial Computer Vision
Harl, Maximilian, Herchenbach, Marvin, Kruschel, Sven, Hambauer, Nico, Zschech, Patrick, Kraus, Mathias
In recent years, large pre-trained deep neural networks (DNNs) have revolutionized the field of computer vision (CV). Although these DNNs have been shown to be very well suited for general image recognition tasks, application in industry is often precluded for three reasons: 1) large pre-trained DNNs are built on hundreds of millions of parameters, making deployment on many devices impossible, 2) the underlying dataset for pre-training consists of general objects, while industrial cases often consist of very specific objects, such as structures on solar wafers, 3) potentially biased pre-trained DNNs raise legal issues for companies. As a remedy, we study neural networks for CV that we train from scratch. For this purpose, we use a real-world case from a solar wafer manufacturer. We find that our neural networks achieve similar performances as pre-trained DNNs, even though they consist of far fewer parameters and do not rely on third-party datasets.
- Europe > Germany > Bavaria > Middle Franconia > Nuremberg (0.14)
- North America > United States > Utah > Salt Lake County > Salt Lake City (0.04)
- North America > United States > New York (0.04)
- (7 more...)
- Research Report (1.00)
- Overview (0.93)
- Energy > Renewable > Solar (0.86)
- Information Technology (0.68)
- Law (0.54)
'High-tech' robot on Russian TV was man in suit: report
MOSCOW - Russian media say a contraption presented by Russian state television as a high-tech robot was in fact a man in a commercially available robot costume. The footage was shot at a high-tech show in the city of Yaroslavl that opened last Tuesday, featuring "Boris the Robot." Forum organizers used Boris to enliven the event, having him dance to a pop song. But a crew for Russian state television apparently thought Boris was real, and used footage of him dancing and speaking as an example of Russian technological prowess. Online TJournal noted the lack of sensors, human-like movements and other discrepancies, and revealed that Boris was in fact a human clad in a costume sold under the name Alyosha by the Russian company Show Robots.
- Europe > Russia > Central Federal District > Yaroslavl Oblast > Yaroslavl (0.32)
- Europe > Russia > Central Federal District > Moscow Oblast > Moscow (0.32)
'Hi-tech robot' called Boris shown dancing on Russian state television has been ridiculed
A Russian'robot' that danced on live state television has been revealed as a scam. Boris the'hi-tech robot' was said to be capable of elaborate movements including dancing, but viewers were immediately sceptical of his smooth moves. It has now surfaced that the bot was actually just a costume worn by an actor and not the ultra high-tech piece of machinery it was claimed to be. A Russian'robot' that danced on live state television has been revealed as a scam. Boris the'hi-tech robot' was said to be capable of elaborate movements including dancing, but viewers were immediately sceptical of his smooth moves.
Whoever leads in AI will rule the world!- Putin to Russian children on Knowledge Day
Vladimir Putin spoke with students about science in an open lesson on September 1, the start of the school year in Russia. He told them that "the future belongs to artificial intelligence," and whoever masters it first will rule the world. "Artificial intelligence is the future, not only for Russia, but for all humankind. It comes with colossal opportunities, but also threats that are difficult to predict. Whoever becomes the leader in this sphere will become the ruler of the world," Russian President Vladimir Putin said.
- Asia > Russia (1.00)
- Europe > Russia > Central Federal District > Yaroslavl Oblast > Yaroslavl (0.07)
- Africa > Cameroon > Gulf of Guinea (0.07)
- Government > Regional Government > Europe Government > Russia Government (0.84)
- Government > Regional Government > Asia Government > Russia Government (0.84)
The threat of killer robots
Artificial intelligence (AI) has a growing number of applications in the security and military areas. It facilitates manoeuvres in the field, and can save lives when things go wrong. It also boosts the performance of armies by providing robot allies to combat forces. According to some experts, Lethal Autonomous Weapons Systems (LAWS) are creating a "Third Revolution" in warfare, after gunpowder and nuclear weapons. It is time we start worrying about the day when armies of robots are capable of conducting hostilities with full autonomy, without humans to command them.
- Asia > Russia (0.31)
- Asia > China (0.14)
- North America > United States > Missouri (0.05)
- (4 more...)
- Government > Military > Army (0.50)
- Government > Regional Government > North America Government > United States Government (0.47)
- Government > Regional Government > Asia Government (0.47)
Putin warns: AI will be the ultimate weapon for world domination… (and Google is working on it)
The Russian president has become the latest person to warn of the dangers of artificial intelligence (AI), actually predicting that whoever masters the technology first can rule the world. Addressing students last week, Vladimir Putin said that there are legitimate concerns about AI and that its development will produce "colossal opportunities and threats that are difficult to predict now." Going further, Putin warned that "the one who becomes the leader in this sphere will be the ruler of the world." He added: "Artificial intelligence is the future, not only for Russia but for all humankind," according to Russia Today. Putin added that he does not want to see the technology "monopolized," and added that Russia would share it with the world if Moscow develops advanced AI first.
- Asia > Russia (1.00)
- Europe > Russia > Central Federal District > Moscow Oblast > Moscow (0.25)
- North America > United States > Rhode Island > Providence County > Providence (0.05)
- Europe > Russia > Central Federal District > Yaroslavl Oblast > Yaroslavl (0.05)
- Government > Regional Government > Europe Government > Russia Government (0.72)
- Government > Regional Government > Asia Government > Russia Government (0.72)